Exploration of very large data sets: The CiTree algorithm

Control variables and command history recorded during normal operation of industrial processes are routinely stored in databases for later analysis. These databases constitute a potentially precious source of information that can be extremely useful from a commercial and strategic perspective. However, the extraction of information from a database is often a non-trivial task, requiring the cooperation of various disciplines, in what has now become a field of research in itself, known as ‘data mining’. One of the first tasks for the data miner is to summarize the complexity of the data into a number of distinct clusters, which represent ‘interesting’, often unexpected, behavior patterns of the process under analysis. Powerful computers and efficient clustering algorithms are now available; nonetheless limits are typically exceeded when mining massive databases such as those arising from industrial processes. Therefore, in order to uncover the potentially useful information held by large bodies of data, new clustering algorithms are needed that directly address the problem of size. We present here one such algorithm, which has been successfully applied to a variety of industrial processes. It yields a hierarchical structure of the clusters present in the process; thus providing a detailed representation of the relationships amongst sample units. We also show, as an example, the application of the algorithm to a real case study, which resulted in the extraction of useful information. Key-Words: Data Mining, Optimization, Model, Industrial, Clustering.